An Approach Integrating Simulation and Q-learning Algorithm for Operation Scheduling in Container Terminals

In this paper, a method integrating Q-learning algorithm and simulation technique is proposed to optimize the operation scheduling in container terminals. Firstly Q-learning algorithms for yard cranes and yard trailers are designed to obtain the optimal scheduling strategy of yard cranes and yard trailers. Then Q-learning is combined with simulation to develop an integrating scheduling model includes all stages of operation process. In this method, simulation model is used to construct the system environment, Q-learning algorithm is used to learn the optimal dispatching rules for equipments, and the optimal scheduling scheme is obtained by the interaction of Q-learning algorithm and simulation environment. Finally, numerical tests are used to illustrate the validity of the proposed method.